MULTIMODAL IMAGING OF LANGUAGE PERCEPTION Doctoral Dissertation

OF DOCTORAL DISSERTATION AALTO UNIVERSITY SCHOOL OF SCIENCE AND TECHNOLOGY P.O. BOX 11000, FI-00076 AALTO http://www.aalto.fi Author Johanna Vartiainen Name of the dissertation Multimodal imaging of language perception Manuscript submitted 09.03.2010 Manuscript revised Date of the defence 04.06.2010 Monograph x Article dissertation (summary + original articles) Faculty Faculty of Information and Natural Sciences Department Department of Biomedical Engineering and Computational Science Field of research Engineering Physics, Biomedical Engineering Opponent(s) Prof. Steve Small Supervisor Prof. Risto Ilmoniemi Instructor Academy Prof. Riitta Salmelin Abstract This Thesis draws together several lines of research by examining language perception in the same individuals using three neuroimaging methods: magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and electroencephalography (EEG). The MEG experiments conducted in this Thesis demonstrated that neural processing of written and spoken words converges to the superior temporal cortex following initial modality-specific analysis. In both reading and speech perception, the superior temporal cortex is involved in processing word meaning at ~250-450 ms in the left hemisphere and after ~450 ms bilaterally. The data thus support a view of a largely shared semantic system in auditory and visual language perception, in line with the assumption that reading acquisition makes use of the neural systems originally developed for speech perception during evolution and in individual language development. The MEG experiments on reading morphologically complex words showed that the left superior temporal activation was enhanced for the morphologically complex words at ~200-700 ms. The results suggest that the majority of inflected words in the highly inflected Finnish language are represented in a decomposed form and that the decomposition process requires additional neural resources. Only very high-frequency inflected words may acquire full-form representations. The MEG results on parafoveal preview in reading indicated that neural processing of written words in the left hemisphere is affected by a preview of words in the right visual field. The underlying neural mechanism may facilitate reading of connected text in natural conditions. In a direct comparison, MEG and fMRI showed diverging activation patterns in a reading task although the same individuals were performing the same task. Based on the similarity of the EEG responses recorded simultaneously with both MEG and fMRI, the participants were performing the task similarly during the two recordings. The divergent MEG and fMRI results cannot be attributed to differences in the experimental procedures or language since these factors were controlled. Rather, they are likely to reflect actual dissimilarities in the way neural activity in a high-level cognitive task is picked up by MEG evoked responses and fMRI signals.